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Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .1 7(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1 .1 14. Applicant's submission filed on 8/22/06 
has been entered. 

Response to Amendment 

The amendment received 8/22/06 has been entered in full. 

Response to Arguments 

Applicant's arguments filed 8/22/06 have been fully considered but they are not 
persuasive. Specifically the Applicant argues that Naigaishi does not teach or suggest 
"proposing to the subject an aesthetic feature for affecting the at least one external body 
condition based, at least in part, on the analyzing of the external body condition [via the 
three dimensional image]," (brackets added by the Applicant). The Examiner has 
pointed out that Naigashi does teach proposing to the subject an aesthetic feature 
(makeup) for affecting the at least one external body condition (skin) base, at least in 
part, on the analyzing of the external body condition. The Examiner does not rely upon 
Naigaishi in order to teach that the analysis is via a three dimensional image, but 
instead relies upon Furuta. Furthermore the Applicant argues that Furuta does not teach 
the "proposing" feature of claim 20. The Examiner however does not rely upon Furuta to 
disclose this limitation but rather relies on Naigaishi. The Applicant then goes on to 
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argue that the Examiner is using impermissible hindsight in order to combine Furuta and 
Naigaishi because Furuta deals with a beauty simulation while Naigaishi is dealing with 
a system to propose a beauty product to the user. In response to applicant's argument 
that the examiner's conclusion of obviousness is based upon improper hindsight 
reasoning, it must be recognized that any judgment on obviousness is in a sense 
necessarily a reconstruction based upon hindsight reasoning. But so long as it takes 
into account only knowledge which was within the level of ordinary skill at the time the 
claimed invention was made, and does not include knowledge gleaned only from the 
applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 
1392, 170 USPQ 209 (CCPA 1971). The application of Furuta and Naigaishi both deal 
with using image processing for beauty consultation purposes. While Naigaishi does not 
explicitly disclose that a beauty simulation is carried out with the proposed makeup, it is 
taught that beauty simulations are well known in the field of image processing (see 
paragraph 5 of the machine translation for the "lip color advice system and approach" 
which carries out simulation of the approach which suited the man is indicated). 
Furthermore both inventions are pointed to the same end of selling beauty products to a 
consumer. Thus based on the teaching of Naigaishi and Furuta it would be obvious to 
one of ordinary skill in the art to combine the two in order to allow an individual to be 
able to discern how they would look with the proposed makeup applied. The motivation 
for doing so would be to allow for a more realistic presentation of the product with the 
hope that it would sell more of the product. Thus the rejection of claim 20 is maintained. 
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Regarding claims 33-36, the Applicant uses the same "proposing" argument 
which the examiner has already addressed above. 

Applicant's arguments with respect to claims 28-31 , and 37 have been 
considered but are moot in view of the new grounds of rejection. 



DETAILED ACTION 

Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 2-6,8-17,19-27,32, 38-39, and 41-44 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Nagaishi JP 2001104050 (machine translation provided) in 
view of Furuta et al. US. Patent Publication 20010037191 (hereinafter "Furuta"). 

Naigaishi discloses a method of enabling an analysis using a facial image (all of 
the following taken from the English translation of the abstract), the method comprising: 
facilitating processing of the facial image to enable a beauty analysis; analyzing the at 
least one external body condition via the image (the condition of the skin is analyzed); 
and proposing (advice is given as to the makeup for the subject to wear) to the subject 
an aesthetic feature for affecting the at least one external body condition based, at least 
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in part, on the analyzing of the external body condition (based on the analysis of the 
face). 

Naigaishi does not explicitly disclose that the method uses a three-dimensional 
facial image and thus does not disclose facilitating construction of a three-dimensional 
facial image using at least one captured image of a subject's face. 

Furuta discloses a three-dimensional beauty simulation client-server system 
which is capable of handling a users face in a three-dimensional fashion and of 
providing more realistic beauty simulations (first sentence of the abstract). As seen in 
figure 3 multiple cameras are disclosed (1a, 1b) and the images are sent to the 3D face 
model generating unit 2. Alternatively if the user has a camera-equipped computer or 
internet cellular phone, a plurality of images obtained therefrom may be sent to the 
server, enabling the server 10 (figure 1) to construct a 3D face model (paragraph 32). 
Also disclosed is the makeup simulation unit 3 that allows simulations of makeup, 
cosmetic surgery, clothing, perfume, accessories, hair style, etc. based on the 3D 
information (paragraph 41). Using this simulation the consumers face may be 
reproduced with the new makeup style, etc. (paragraph 38). Furuta discloses that there 
is a problem with prior art systems in which the two dimensional image processing did 
not appear realistic, thus proposes making the system three dimensional (paragraphs 5 
and 6 of page 1). 

Naigaishi and Furuta are analogous art because they are from the same field of 
endeavor of image processing for beauty consultation purposes. 
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At the time of the invention it would have been obvious to one of ordinary skill in 
the art to modify Naigaishi's two-dimensional system to make it three-dimensional 
according to the teachings of Furuta. The motivation for doing so would be to provide 
realistic beauty simulations that allow the consumer to see how they would look with the 
product. Thus it would have been obvious to one of ordinary skill in the art at the time of 
the invention to combine Naigaishi and Furuta to obtain the invention of claim 20. 

Regarding claims 2-5, and 21-24 Furuta discloses that the means of the 
invention may be realized through software (paragraphs 161-162). Furthermore the 
server consists of a data control unit for storing information (10c figure 1). 

Regarding claim 6, makeup is a beauty product. 

Regarding claim 8, Furuta discloses that using morphing technology can enable 
one to resemble one's favorite model, and the user can learn what percentage of the 
image comprises her own features and what percentage comprises the models features 
(paragraph 41). Furthermore a simulation in which the level of beauty and degree of 
aging of the face are assessed can be carried out (paragraph 43). 

Regarding claim 9, Furuta discloses that the makeup simulation drawing software 
uses a method in which the face as a whole is made up by applying makeup to 
individual parts of the face, and the part of the face is selected by the user including lips, 
eyebrows etc. In this method the sought makeup style is pasted onto the image 
(paragraph 57). 

Regarding claim 10-11, as discussed Furuta discloses makeup. 
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Regarding claim 12, Furuta discloses that it is possible to view the face from 
various angles, as if the camera viewpoint had continuously moved (paragraph 104). 

Claim 13 is dependent on claim 1 (rejected above by Furuta) and recites 
instructing the subject on how to transmit the at least one captured image to a location 
remote from the subject, wherein the three-dimensional representation is constructed at 
the remote location. Furuta discloses that the consumer may send images to the server 
to construct 3D face data (paragraph 32). Furuta does not explicitly disclose instructing 
the subject on how to transmit the image. It is obvious that the internet site used to 
submit the images would provide some instruction as to how to submit the images, so 
that the user could avoid frustration of trying to find out for themselves how to do it, and 
thus experience a pleasant shopping experience and want to return in the future for 
more shopping. 

Claim 14 discloses prompting the subject to capture the at least one image of the 
subjects face. Furuta discloses that images may be captured (paragraph 32). It would 
be obvious to prompt the user before taking the image so that an image of the user with 
his eyes closed or in an awkward position would not be used for the later analysis. 

Regarding claim 15, Furuta discloses that the processing can take place through 
the internet to allow for analyzing the image at a location remote from the subject. 

Regarding claim 1 6, as seen in figure 3 a plurality of cameras (1 a 1 b) are used to 
capture a plurality of facial images. 
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Regarding claim 17, Furuta discloses the facial image generator beginning with 
paragraph 108 in which a three dimensional structure (frame) is used in coordination 
with the captured images. 

Regarding claim 19, the evaluation can take place over an internet connection 
(1 1 figure 1 ) allowing for analyzing at a location remote from the subject. 

Regarding claim 25, Furuta discloses that using morphing technology 
information to enable one to resemble one's favorite model may be obtained, for 
example intermediate images resembling a cross between the user and their favorite 
model may be created (paragraph 41). 

Claim 26 is similar to claim 1 , except it is directed to a computer readable 
medium. Thus the same arguments used for the rejection of claim 1 apply equally to the 
rejection of claim 26. 

Claim 27 is similar to claim 1 , except it is directed to a system. Thus the same 
arguments used for the rejection of claim 1 apply equally to the rejection of claim 27. 

» 

Regarding claims 32 and 39, Furuta discloses a makeup simulation unit 3 using 
the facial model. 

Regarding claim 38, Nagaishi discloses analyzing the face image and outputting 
information showing the condition of the skin, and since advice is given based on the 
condition of the skin the extent and severity of the skin condition must be determined. 

Regarding claims 41-42 Furuta discloses that each face has its own light and 
dark effects which are taken into account in the simulation (paragraph 45). 
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Regarding claims 43-44, if a three-dimensional image is viewed straight on then 
it will appear two dimensional. 

3. Claims 28-31 , and 40 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nagaishi JP 2001 104050 in view of Furuta et al. US. Patent Publication 
20010037191 (hereinafter "Furuta") and further in view Nobori et al. Image Synthesis 
System Using 3D Model-based Coding Simulates Facial Expressions and Aging 
(hereinafter "Nobori", cited in the IDS 8/22/06). 

Regarding claim 28 Naigaishi nor Furuta explicitly disclose facilitation 
construction of a three-dimendisonal facial image using at least one captured image of a 
subject's face comprises applying the at least one captured image in a virtual manner 
on a three-dimensional frame. 

Nobori discloses a fitting algorithm using a polygonal wireframe model to obtain a 
synthesized 3d face model (section 2). This provides a practical approach for forming 
the synthesized 3d model (conclusion). 

Naigaishi, Furuta, and Nobori are analogous art because they are all from the 
same field of endeavor of image processing for beauty consultation purposes. 

At the time of the invention it would have been obvious to one of ordinary skill in 
the art to make the 3d model as discussed by Furuta using a 3d frame. The motivation 
is that it is a practical way of doing so. Regarding claim 29, as seen in figure 1 Nobori 
discloses a virtual wire mesh. 
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Regarding claim 30-31, Nobori discloses generating the frame based on stored 
and inputted information (figure 1). 

Regarding claim 40, Nobori discloses adding a dimensional effect to the three- 
dimensional facial image (section 2). 



4. Claims 33-36 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nagaishi JP 2001 104050 in view of Furuta et al. US. Patent Publication 20010037191 
(hereinafter "Furuta") and further in view of Cosatto et al. USPN 6,504,546 (hereinafter 
"Cosatto"). 

Nagaishi nor Furuta explicitly disclose enabling a calibration of the three- 
dimensional facial image to address a perceived difference between the three- 
dimensional facial image and the subjects face. 

Cosatto discloses a method for modeling three-dimensional objects wherein 
using color calibration periodically the appearances of facial features can be tracked. In 
this manner reliable measurements of the outline of the facial features are obtained (col. 
8 lines 15-41). 

Nagaishi, Furuta, and Cosatto are all analogous art because they are all from the 
same field of endeavor of facial imaging. 

At the time of the invention it would have been obvious to one of ordinary skill in 
the art to enable calibration within the inventions of Nagaishi and Furuta as taught by 
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Cosatto. The motivation for doing so is that changes in the appearances of the facial 
features can be easily tracked. Thus it would have been obvious to one of ordinary skill 
in the art at the time of the invention to combine Naigaishi, Furuta, and Cosatto in order 
to obtain the invention as specified in claim 33. 

Regarding claim 34, as discussed Cosatto discloses color calibration. 

Regarding claim 35, Furuta discloses a makeup simulation unit 3 using the facial 

model. 

Regarding claim 36, Cosatto discloses the color-calibrated three-dimensional 
facial image and Furuta discloses a makeup simulation unit 3 using the facial model. 

5. Claim 37 is rejected under 35 U.S.C. 103(a) as being unpatentable over Nagaishi 
JP 2001104050 in view of Furuta et al. US. Patent Publication 20010037191 
(hereinafter "Furuta") in view of Cosatto et al. USPN 6,504,546 (hereinafter "Cosatto"), 
and further in view of Vinas et al. USPGPUB 2003/0164955 (hereinafter "Vinas"). 

Cosatto discloses color calibration and tracking changes in the appearances of 
facial feature, however does not explicitly disclose allowing the user to manually 
calibrate the image. It is well known to allow manual color calibration in image 
processing. Vinas discloses that if a user is very sensitive to color issues then he may 
manually initatiate a color calibration process at any time (paragraph 46). 

Cosatto and Vinas are analogous art because they are from the same field of 
endeavor of color calibration. 



Application/Control Number: 10/024,615 Page 12 

Art Unit: 2624 

At the time of the invention it would have been obvious to one of ordinary skill in 
the art to combine Cosatto and Vinas to allow for manual color calibration since it would 
give the user more control of the process if they are color sensitive. Thus it would have 
been obvious to one of ordinary skill in the art to combine Nagaishi, Furuta, Cosatto, 
and Vinas to obtain the invention as specified in claim 37. 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to John B. Strege whose telephone number is (571) 272- 
7457. The examiner can normally be reached on Monday-Friday between the hours of 
8-5. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on (571) 272-7453. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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